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ABSTRACT 



The jackknife, as refined by J. Tukey (1958) , is a valuable 
tool for the internal replication of a study. The jackknife statistic is 
particularly useful with small sample sizes. Large samples are labor 
intensive, and other methods better address this situation. The jackknife 
procedure involves the use of a single sample drawn from a normally 
distributed population. The jackknife procedure is a general method for 
reducing the bias in an estimator while providing a measure of the variance 
of the resulting estimator by sample reuse. The essence of the jackknife 
approach is to partition out the impact of a particular siibset of the data on 
an estimate derived from the total sample. The method attempts to determine 
if any one case or group of cases exerts an inappropriate influence on the 
overall statistic of interest. To illustrate the value of the jackknife, an 
example is presented that uses actual educational research data. The study 
(B. White and L. Daniel, 1999) concerned career motivations of persons 
planning to teach. (Contains 4 tables and 13 references.) (SLD) 
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Result Generalizability and Detection of Discrepant Data Points: 

Illustrating the Jackknife Method 

The dictionary defines replicate as “to duplicate, copy, or repeat.” Replication in 
research enhances credibility of the research process by eliminating that which 
sometimes contaminates research results; namely suspicion that the sample employed 
may be biased. Replication of research is the way scientists verify their results (Frymier, 
Gansneder & Robertson, 1989). Kessen (1960) termed replication as an elemental 
principle of competent research; however, replication of research studies is too often the 
exception rather than the rule. Examining psychological studies. Smith (1970) identified 
reasons why researchers fail to replicate; (a) lack of funds, (b) lack of time, (c) lack of 
availability of a comparable group of subjects, (d) development of new research interests, 
(e) desire to publish, (f) ego involvement with the data, and (g) the reluctance of some 
journals to publish replication studies. Frymier et. al (1989) stated, “Replication requires 
precise duplication of methods, instruments, and time lines if the concept is honored.” (p. 
3) Precision of this kind, though possible in the physical sciences, presents many 
obstacles in behavioral science research. Still, within education research, replication is 
essential to (a) check that the sample is not overly biased, and (b) verify that the results of 
the research are stable. 

The purpose of all research is to generalize findings to the entire population, or to 
a population of interest. Therefore, it is imperative that research results, especially in 
education, be reported with confidence. The thinking researcher will take whatever steps 
necessary to produce credible results that can be generalized to the population of interest, 
including selection and reporting of the most significant data treatments and tests. One of 
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the recognized triumvirates for reporting statistical results includes (a) statistical 
significance, (b) effect size, and (c) result replicability. There are philosophical issues 
even among the former. Larry Daniel’s work concerning the misuse of statistical 
significance testing (Daniel, 1998) should be referenced for a clear discussion of these 
issues. However, it is the latter of the three, replicability that is so frequently ignored. 
Problems with replication 

The reticence of researchers to replicate their studies is understandable. Most 
statistical methods in use today were developed between 1800 and 1930 when 
computation was slow and expensive. Many researchers were trained under the 
assumption that replication was cost prohibitive and time consuming. A review of 20 
standard textbooks on experimental design and methodology found that 12 did not list 
replication in their index, while 7 gave passing reference to the topic, and only Sidman 
(1960) treated the subject at length (Smith, 1970). However, with the advent of high- 
speed computers, and user-friendly statistical software, new methods for computing 
statistical analysis are fast and cost-efficient (Diaconis & Efron, 1983). 

Computers have also allowed researchers to develop methods for another type of 
replication: internal replication. The more commonly known methods include cross- 
validation, the bootstrap by Efron, and the jackknife (Quenouille, 1949; Tukey, 1958). 
This paper concerns the latter: the jackknife 
Using internal replication 

The jackknife, as refined by Tukey (1958), is a valuable tool for internal 
replication of a study. The jackknife statistic is particularly useful with small sample 
sizes. Large samples are labor intensive, and other methods better address this situation. 
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The jackknife procedure involves the use of a single sample drawn from a normally 
distributed population (Schumacker, in press). The jackknife statistic is a general method 
for reducing the bias in an estimator while providing a measure of the variance of the 
resulting estimator by sample reuse. The essence of the jackknife approach is to partition 
out the impact of effect of a particular subset of the data on an estimate derived from the 
total sample (Crask & Perrault, 1977). The method attempts to determine if any one case 
or group of cases exerts an inappropriate influence on the overall statistic of interest (e.g., 
effect size, or R^). With a single sample of data, the jackknife computes a new sample 
statistic for each sample size of K-1. The total number of psuedovalues (created by the 
jackknife) will equal the original sample size. The precise calculations for this statistic 
will later be discussed. 

Data Example 

To illustrate the value of the jackknife, a discussion using actual educational 
research data follows. The data (White & Daniel, 1999), used by permission, concerns 
career motivations of persons planning to teach. Initially, a regression analysis was 
completed on a randomly selected sample of White and Daniel’s complete data set. The 
working sample can be viewed in Table 1 . While the jackknife procedure can be used to 

Insert Table 1 About Here 

determine the stability of several statistics, the statistic of interest for the present study is 
the effect size, or R^. It should be noted that the jackknife method is useful for 
determining the stability of several statistics (e.g., weights). However, it should also be 
noted that the effect sizes that are evaluated in statistical significance testing are more 
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stable than the weights used to derive these effects (Thompson, 1994). Hence, the 
statistic of interest in all examples here mentioned is the effect size measure. An 
examination of Table 2 shows the for the initial regression was .391, an appreciable 
effect size. In order to determine the stability of this effect, the jackknife approach to 
internal replication was applied. 



Insert Table 2 About Here 



The calculation for the jackknife procedure is best addressed in Crask and Perrault’s 
pivotal work (1977). The calculation is simply: 

Ji = k(0') - (k-1)0i 

Where Ji = Statistic of interest computed without the first (2"‘*, etc.) case 
K = number of cases 

0 " = of regression all cases 

0i = R^ of regression without the first (2"‘*, etc) case 

i = the given jackknife replication (i to K) 

The procedure itself is simple. Each time the statistic of interest is computed, one case 
(or subset of cases) is eliminated. The statistic is computed K-1 times until each subset 
of data has been analyzed. The result is a pool of pseudovalues of the statistic from 
which the jackknife estimate can be calculated. 



Insert Table 3 About Here 



The remaining portion of the jackknife calculation may be employed to find the 
individual pseudovalues. Each of these values constitutes the error between the actual 
statistic of interest and the jackknifed estimate. The sum of the errors is divided by the 
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number of cases, k (S Ji /N). Further, the researcher must obtain the mean and standard 

deviation of the jackknife calculations ((k- 1 ) 0 1 ). In order to create a confidence 

interval, the critical value for the 95% confidence interval (1.96) is multiplied by the 
standard deviation. This value, added and subtracted from the mean, creates the 
confidence interval. The thoughtful researcher hopes the original statistic of interest falls 
between the confidence delimiters. 

For the present example using the subset of White and Daniel’s data (1999), the 
original = .391. Table 2 contains the recorded values for the subsequent 
regressions. The results for the example calculations can be found on table 3. The 
jackknifed value (.308) 

Insert Table 4 About Here 

resembles the original statistic (.391) and falls within the 95% confidence interval [4.74, 
-1 .25]. The conclusion can be drawn that a certain stability exists within the research 
result estimator. A researcher could then report his/her results with an air of certainty and 
gain wider acceptance and credibility. 

Conclusion 

It is important to note that there is no substitute for actual, careful replication of a 
research study. Nothing adds more certainty to a result than its continued confirmation 
by further research studies. However, as previously mentioned, there are myriad reasons 
why actual replication is frequently improbable or impossible. So what is the answer? 
Researchers must begin to employ every tool at their disposal to add credibility to their 
work. In the field of education, the need for accuracy in reporting research is critical as 
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these studies are so frequently used in decision-making. Why then do modem 
researchers neglect this vital need? 

The jackknife method is labor intensive. It is only useful with small samples, 
and causes a researcher to regress many subsets of his/her data and calculate 
pseudovalues and confidence intervals. Occasionally, the confidence interval appears 
unrealistic because the researcher has forced the assumption of normality on data that 
may not be normally distributed. Internal replication is possible with the use of 
computers but still requires a measure of time and effort on the part of the researcher. 

Even so, many tools are available to aid in the process of internal replication. 
Diaconis and Efron (1983) clearly illustrate the advantage of computer intensive research 
models. A researcher needing 40 regression analyses for a jackknife procedure need only 
“point and click” 40 times in a statistical package like SPSS. In light of the advantages 
computers bring to research, it is at least the duty of researchers to utilize them for the 
betterment of research. Internal replication studies of any kind, regardless of labor 
intensity are still easier, faster, and less expensive than the careful, accurate, and precise 
replication of a research study. And though internal replication is never as good as actual 
replication, it is still better than the replicability evidence that most researchers provide: 
nothing. 
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Table 1 



Random Subset of White and DanieTs (1999) Data 



Used by Permission 



CASE MAJOR 


YRBIRTH 


SERVINTR 


SECURITY 


CONTINUE 


BENECOMP 


STIMULAT 


FILTER 


1 


ELEM ED 


77 


4 .73 


2.77 


4.00 


3.55 


4.00 


1 


2 


ENGLISH 


75 


3.47 


2.46 


3.40 


2 . 73 


3.00 


1 


3 


HISTORY 


76 


3.87 


1 . 62 


3.80 


2.36 


3.56 


1 


4 


ELEM ED 


76 


4.00 


1.46 


3.30 


2 . 91 


2.33 


1 


5 


ELEM ED 


77 


3.73 


2.23 


2.50 


2.00 


2 .44 


1 


6 


SPEECH 


76 


4.20 


2.23 


4.20 


3.18 


4.00 


1 


7 


ELEM ED 


60 


4.53 


2.23 


3.70 


4.09 


3.78 


1 


8 


LIB SCI 


34 


3.80 


3.00 


2.50 


2.18 


4.22 


1 


9 


ELEM ED 


73 


3.87 


2.31 


3.70 


3.82 


2.89 


1 


10 


ELEM ED 


72 


3.53 


1.85 


4.00 


2.27 


2 . 67 


1 


11 


ELEM ED 


70 


4.20 


2.46 


4 . 00 


3.18 


2.33 


1 


12 


ELEM ED 


76 


3.53 


2.31 


3.60 


3.18 


3.11 


1 


13 


SPEC ED 


72 


3.60 


2.00 


3.70 


2.36 


2.78 


1 


14 


MUSICED 


74 


5.00 


2.00 


4 . 10 


2.82 


3.33 


1 


15 


ELEM ED 


74 


3.33 


2.85 


2.20 


2.73 


4 . 44 


1 


16 


SP PATH 


74 


4.00 


1.85 


3.90 


2 . 64 


3.33 


1 


17 


ENGLISH 


75 


3.40 


2.15 


3.60 


2.73 


2 . 67 


1 


18 


ELEM ED 


67 


4 .80 


1.31 


4 . 40 


2.73 


2.67 


1 


19 


ENGLISH 


75 


4.27 


1 . 62 


4 . 10 


2.09 


3.22 


1 


20 


ELEM ED 


77 


4.33 


2.15 


3.90 


3.00 


3.33 


1 


21 


SCIENCE 


56 


4.33 


1.85 


3.90 


2.82 


3.00 


1 


22 


SPEC ED 




4 . 67 


3.77 


3.40 


4.55 


3.89 


1 


23 


SCIENCE 


73 


3.00 


3.08 


2.00 


1.82 


3.22 


1 


24 


ELEM ED 


76 


4 . 67 


2.00 


4.30 


3.36 


3.67 


1 


25 


MUSIC 


77 


4.00 


1.46 


4.20 


3.45 


3.22 


1 


26 


ELEM ED 


77 


3.53 


1.85 


3.60 


3.55 


3.00 


1 


27 


ELEM ED 


72 


3.80 


1.31 


4 . 10 


3.45 


3.00 


1 


28 


HISTORY 


75 


3.73 


1 . 54 


4 . 40 


1.91 


2.22 


1 


29 


ELEM ED 


77 


4.20 


2.31 


3.90 


3.18 


2.89 


1 


30 


MUSIC 


73 


3.73 


1 . 62 


3.50 


2 .73 


3.22 


1 


31 


SP PATH 


77 


3.53 


2.15 


3.60 


3.82 


2.89 


1 


32 


ELEM ED 


71 


4.53 


3.38 


2.60 


3.00 


4.22 


1 


33 


ELEM ED 


77 


4.27 


2.31 


3.90 


3.18 


2.89 


1 


34 


ELEM ED 


69 


4.80 


1 . 77 


4 . 50 


3.55 


3.78 


1 


35 


SP PATH 


77 


4 .87 


2.77 


2.00 


3.18 


3.67 


1 


36 


ELEM ED 


75 


4.13 


1.62 


4.00 


2 . 91 


3.11 


1 


37 


ELEM ED 


77 


4 .47 


1.92 


4.10 


3.09 


2.89 


1 


38 


SP PATH 


75 


3.40 


3.31 


2.50 


3.09 


3.89 


1 


39 


BUS ED 


72 


4.20 


4.08 


4.30 


2.27 


2.56 


1 


40 


ELEM ED 


70 


5.00 


3.31 


4.90 


4.09 


4.56 


0 



O 
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Table 2 

Regression analysis with all cases present 



Model 


R 


R Square 


Adjusted R 
Square 


Std. Error of 
the Estimate 


1 


.626 


.391 


.322 


.4245 
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Table 3 

Results of K-1 regression analyses 



Subjcct/Statistic 



Pseudo-values 



cases 


.391 




1. 


.369 


14.39 


2. 


.387 


15.09 


3. 


.393 


15.33 


4. 


.406 


15.83 


5. 


.407 


15.87 


6. 


.400 


15.60 


7. 


.381 


14.86 


8. 


.387 


15.09 


9. 


.396 


15.44 


10. 


.388 


15.13 


11. 


.395 


15.41 


12. 


.401 


15.64 


13. 


.384 


14.98 


14. 


.404 


15.76 


15. 


.394 


15.37 


16. 


.392 


15.29 


17. 


.385 


15.02 


18. 


.416 


16.22 


19. 


.395 


15.41 


20. 


.390 


15.21 


21. 


.396 


15.44 


22. 


.373 


14.55 


23. 


.330 


12.87 


24. 


.374 


14.59 


25. 


.399 


15.56 


26. 


.402 


15.68 


27. 


.402 


15.68 


28. 


.386 


15.05 


29. 


.391 


15.25 


30. 


.388 


15.13 


31. 


.408 


15.91 


32. 


.407 


15.87 


33. 


.392 


15.29 


34. 


.364 


14.19 


35. 


.537 


20.94 


36. 


.391 


15.25 


37. 


.393 


15.33 


38. 


.394 


15.37 


39. 


.391 


15.25 


40. 


.337 


13.14 
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Table 4 

Jackknife result for example data set 





Results 


S of differences 


12.76 


E/K 


.319 


SD 


1.21 


Confidence Interval 


[4.74, -1.25] 



Conclusion Stable Result 

*The original effect (.391) falls within the 95% confidence interval 
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